Efficient Assessment of Asr Systems by Using Subsets of a Test Database

In this paper, assessment of ASR systems with a limited set of speech data selected from a larger testing corpus was studied for connected Dutch digits. Three methods of data selection were applied, namely random, knowledge-based, and datadriven selection. The goal of this study was to find out whether reliable assessment of speech recognition systems can be achieved by using a small sample of the testing corpus. The results are presented in terms of the confidence interval of the mean value calculated for the recognition scores. It appeared that the method of data selection used in this experiment did not contribute significantly to minimize the range of the confidence interval with respect to random selection. Thus, for the speech material presented here, random selection can be successfully applied to obtain a satisfactory assessment even with relatively small subsets of the testing corpus.